---
id: tutorial-llm-application-example
title: Building Your LLM Application
sidebar_label: Building Your LLM Application
---

:::caution

This page is merely an example of a hypothetical LLM chatbot application you might be evaluating. You should definitely replace this example with your own LLM application. For those that want to skip this section, these are the hyperparameters we'll be iterating our LLM application on for this tutorial:

```python
MODEL = "gpt-3.5"
TEMPERATURE = 1.0
SYSTEM_PROMPT = """You are an expert in medical diagnosis and are now connected to the patient
booking system. The first step is to create the appointment and record the symptoms. Ask for specific
symptoms! Then after the symptoms have been created, make a diagnosis. Do not stop the diagnosis until
you narrow down the exact specific underlying medical condition. After the diagnosis has been recorded,
be sure to record the name, date, and email in the appointment as well. Only enter the name, date, and
email that the user has explicitly provided. Update the symptoms and diagnosis in the appointment.
Once all information has been recorded, confirm the appointment."""
```

:::

## Quick Summary

In this section, we will be developing an **Agentic RAG** medical chatbot to record symptoms, diagnose patients, and schedule appointments.

:::info
For the purposes of this tutorial, we'll be using [The Gale Encyclopedia of Alternative Medicine](https://staibabussalamsula.ac.id/wp-content/uploads/2024/06/The-Gale-Encyclopedia-of-Medicine-3rd-Edition-staibabussalamsula.ac_.id_.pdf) as our knowledge base.
:::

## Setting Up

Begin by installing the necessary packages. We'll use `llama-index` as our RAG framework and `chromadb` for vector indexing.

```bash
pip install llama-index llama-index-vector-stores-chroma doc2text chromadb
```

Since our chatbot will be recording patient information, we’ll need a structured way to store it. We'll define a `pydantic` model with the relevant fields (symptoms, diagnoses, and personal information) to ensure that our agent can **accurately store data in the correct format**.

```python
from pydantic import BaseModel
from typing import Optional

class MedicalAppointment(BaseModel):
    name: Optional[str] = None
    email: Optional[str] = None
    date: Optional[str] = None
    symptoms: Optional[str] = None
    diagnosis: Optional[str] = None
```

## Defining Your Chatbot

Next, we'll create a `MedicalAppointmentSystem` class to represent our agent. This class will store all `MedicalAppointment` instances in an `appointments` dictionary, with each key representing a unique user.

```python
class MedicalAppointmentSystem:
    def __init__(self):
        self.appointments = {}
```

As we progress through this tutorial, we'll gradually enhance this class until it evolves into a fully functional medical chatbot agent.

## Indexing Your Knowledge Base

Let's start by building our **RAG engine**, which will handle all patient diagnoses. The first step is to load the relevant medical information chunks from our knowledge base into the system. We'll use the `SimpleDirectoryReader` from `llama-index` to accomplish this.

```python
from llama_index.core import SimpleDirectoryReader

class MedicalAppointmentSystem:
    def __init__(self, data_directory):
        self.appointments = {}
        self.load_data(data_directory)

    def load_data(self, data_directory):
        # Load documents from a directory
        self.documents = SimpleDirectoryReader(data_directory).load_data()
```

Then, we'll use LlamaIndex's `VectorStoreIndex` to embed our chunks and store them in a `chromadb` database. This step is crucial, as our encyclopedia contains over 4,000 pages of dense medical information.

:::tip
Embedding the data in a **vector database** ensures fast and accurate retrieval, even with such a large knowledge base.
:::

```python
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, StorageContext
from llama_index.vector_stores.chroma import ChromaVectorStore

class MedicalAppointmentSystem:
    def __init__(self, data_directory, db_path):
        self.appointments = {}
        self.load_data(data_directory)
        self.store_data(db_path)

    def load_data(self, data_directory):
        ...

    def store_data(self, db_path):
        # Set up the database and store vectorized data
        db = chromadb.PersistentClient(path=db_path)
        chroma_collection = db.get_or_create_collection("medical_knowledge")
        vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
        storage_context = StorageContext.from_defaults(vector_store=vector_store)
        self.index = VectorStoreIndex.from_documents(self.documents, storage_context=storage_context)
```

## Building Your Tools

Finally, we'll create the tools for our chatbot, which includes our **RAG engine** and **function-calling tools** responsible for creating, updating, and managing medical appointments, ensuring the system is both dynamic and interactive.

### Assembling the RAG Engine

In `llama-index`, a RAG engine is abstracted as a `QueryEngineTool`, making it ideal for building agentic applications like this one. This approach also allows you to define additional tools alongside the RAG engine for enhanced functionality.

```python
from llama_index.core.tools import QueryEngineTool

class MedicalAppointmentSystem:
    def __init__(self, data_directory, db_path):
        self.appointments = {}
        self.load_data(data_directory)
        self.store_data(db_path)
        self.setup_tools()
    ...

    def setup_tools(self):
        query_engine = self.index.as_query_engine()
        self.medical_diagnosis_tool = QueryEngineTool.from_defaults(
            query_engine,
            name="medical_diagnosis",
            description="A RAG engine for retrieving medical information."
        )
```

### Function-calling Tools

We'll also need to define the tools for interacting with the medical appointment system, enabling tasks like creating, updating, and confirming appointments. LlamaIndex's `FunctionTool` simplifies this by wrapping functions into callable tools.

```python
from llama_index.core.tools import FunctionTool

class MedicalAppointmentSystem:
    def __init__(self, data_directory, db_path):
        self.appointments = {}
        self.load_data(data_directory)
        self.store_data(db_path)
        self.setup_tools()
    ...

    def setup_tools(self):
        # Configure function tools for the system
        self.get_appointment_state_tool = FunctionTool.from_defaults(fn=self.get_appointment_state)
        self.update_appointment_tool = FunctionTool.from_defaults(fn=self.update_appointment)
        self.create_appointment_tool = FunctionTool.from_defaults(fn=self.create_appointment)
        self.record_diagnosis_tool = FunctionTool.from_defaults(fn=self.record_diagnosis)
        self.medical_diagnosis_tool = QueryEngineTool.from_defaults(
            self.query_engine,
            name="medical_diagnosis",
            description="A RAG engine for retrieving medical information."
        )
```

**Key Tools:**

- `get_appointment_state_tool`: Retrieves the state of a specific appointment.
- `update_appointment_tool`: Updates a property of an appointment.
- `create_appointment_tool`: Creates a new appointment.
- `record_diagnosis_tool`: Records a diagnosis for an appointment.

```python
# Retrieves the current state of an appointment based on its ID.
def get_appointment_state(self, appointment_id: str) -> str:
    try:
        return str(self.appointments[appointment_id].dict())
    except KeyError:
        return f"Appointment ID {appointment_id} not found"

# Updates a specific property of an appointment.
def update_appointment(self, appointment_id: str, property: str, value: str) -> str:
    appointment = self.appointments.get(appointment_id)
    if appointment:
        setattr(appointment, property, value)
        return f"Appointment ID {appointment_id} updated with {property} = {value}"
    return "Appointment not found"

# Creates a new appointment with a unique ID.
def create_appointment(self, appointment_id: str) -> str:
    self.appointments[appointment_id] = MedicalAppointment()
    return "Appointment created."

# Records a diagnosis for an appointment after symptoms have been noted.
def record_diagnosis(self, appointment_id: str, diagnosis: str) -> str:
    appointment: MedicalAppointment = self.appointments.get(appointment_id)
    if appointment and appointment.symptoms:
        appointment.diagnosis = diagnosis
        return f"Diagnosis recorded for Appointment ID {appointment_id}. Diagnosis: {diagnosis}"
    return "Diagnosis cannot be recorded. Please tell me more about your symptoms."
```

## Defining Your Hyperparameters

This part's important, because the whole point of evaluation is to improve on the hyperparameters such as prompts and model used in your LLM application. Throughout this tutorial, we'll be improving on the system prompt, as well as the model used. Here are the variable values:

```python
MODEL = "gpt-3.5"
TEMPERATURE = 1.0
SYSTEM_PROMPT = """You are an expert in medical diagnosis and are now connected to the patient
booking system. The first step is to create the appointment and record the symptoms. Ask for specific
symptoms! Then after the symptoms have been created, make a diagnosis. Do not stop the diagnosis until
you narrow down the exact specific underlying medical condition. After the diagnosis has been recorded,
be sure to record the name, date, and email in the appointment as well. Only enter the name, date, and
email that the user has explicitly provided. Update the symptoms and diagnosis in the appointment.
Once all information has been recorded, confirm the appointment."""
```

## Putting It All Together

Now that we have set up the tools and data systems, it's time to finalize the chatbot agent. We'll use LlamaIndex's `FunctionCallingAgent` to dynamically manage user interactions and choose the appropriate tool based on the input and context. This involves defining the LLM, system prompt, and tool integrations.

```python
from llama_index.core.agent import FunctionCallingAgent
from llama_index.core.llms import ChatMessage
from llama_index.llms.openai import OpenAI

class MedicalAppointmentSystem:
    ...
    def setup_agent(self):
        # use MODEL and TEMPERATURE here
        gpt = OpenAI(model=MODEL, temperature=TEMPERATURE)
        self.agent = FunctionCallingAgent.from_tools(
            tools=[
                self.get_appointment_state_tool,
                self.update_appointment_tool,
                self.create_appointment_tool,
                self.record_diagnosis_tool,
                self.medical_diagnosis_tool,
                self.confirm_appointment_tool
            ],
            llm=gpt,
            prefix_messages=[
                ChatMessage(
                    role="system",
                    # use SYSTEM_PROMPT here
                    content=SYSTEM_PROMPT,
                )
            ],
            max_function_calls=10,
            allow_parallel_tool_calls=False
        )
```

Finally, we'll create an interactive environment where users can engage with the chatbot. This involves configuring input/output, managing conversation flow, and processing user queries.

```python
class MedicalAppointmentSystem:
    ...
    def interactive_session(self):
        print("Welcome to the Medical Diagnosis and Booking System!")
        print("Please enter your symptoms or ask about appointment details. Type 'exit' to quit.")

        while True:
            user_input = input("Your query: ")
            if user_input.lower() == 'exit':
                break

            response = self.agent.chat(user_input)
            print("Agent Response:", response.response)
```

To test your chatbot, run the following code:

```python
if __name__ == "__main__":
    system = MedicalAppointmentSystem(data_directory="./data", db_path="./chroma_db")
    system.interactive_session()
```

Congratulations on building your Agentic RAG application! In the next sections, we’ll explore how to evaluate our medical chatbot, from selecting relevant metrics to iterating on the chatbot's hyperparameters for improved performance.
